Imaging device with a plurality of pixel arrays

ABSTRACT

An imaging device includes a first pixel array arrange to capture a first image and a second pixel array arranged to capture a second image. The first pixel array and the second pixel array face substantially a same direction. The imaging device also includes shutter control circuitry which is coupled to the first pixel array to initiate a first exposure period of the first pixel array to capture the first image. The shutter control circuitry is also coupled to the second pixel array to initiate a second exposure period of the second pixel array to capture the second image. The imaging device also includes processing logic coupled to receive first pixel data of the first image and coupled to receive second pixel data of the second image. The processing logic is configured to generate at least one image using the first pixel data and the second pixel data.

TECHNICAL FIELD

This disclosure relates generally to optics, and in particular but notexclusively, relates to digital imaging devices.

BACKGROUND INFORMATION

Conventional digital imaging devices or cameras have a lens (which mayinclude multiple lens elements) that focuses image light onto an imagesensor that measures the image light and generates an image based on themeasurements. FIG. 1 illustrates a common configuration for a digitalimaging device 100. FIG. 1 includes an image sensor 101 and opticalefficiency lenses 110 disposed over image sensor 101. Optical efficiencylenses 110 function to draw as much light as possible into the pixelsfor measurement. Optical efficiency lenses 110 may be microlensesdisposed over each pixel of image sensor 101. An infrared (“IR”) filter115 may be disposed over optical efficiency lenses 110 and image sensor101 to filter out IR light from being measured by image sensor 101. Lens120 is disposed over image sensor 101 to focus image light 190 onto thepixels of image sensor 101. Lens 120 may include convex and/or concavelens elements 123 that give lens 120 a certain focal length. The focallength of lens 120 may correspond with a Depth of Field. Depth of Fieldrefers to the range of distances in the field of view of an image sensorthat appear to be well focused in an image captured by image sensor 101.

To achieve a given resolution, a conventional digital imaging device mayrequire a certain footprint in a given aspect ratio (e.g. 4:3, 16:9).Conventional digital imaging devices may also have a limited field ofview for a given image capture. Conventional digital imaging devices mayalso be limited to a given number of image captures in a specific amountof time (e.g. 30 frames per second). Some conventional digital imagingdevices are also limited to a given exposure time in a given imagecapture. This may limit the dynamic range of the image captured. Aconventional digital imaging device also typically only has one depth offield in any given image captured because lens 120 has one focusdistance at one time. For example, the foreground of an image may be infocus because it is within the depth of field for that image, but thebackground of the image may be blurred because it was not within thedepth of field for that image. Therefore, a device or method that allowsa digital imaging device to overcome all or some of these limitationswould be advantageous.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the invention aredescribed with reference to the following figures, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified.

FIG. 1 is a side view of a conventional digital imaging device thatincludes a lens focusing image light onto an image sensor.

FIGS. 2A and 2B show example image sensors that include different pixelgroups, in accordance with an embodiment of the disclosure.

FIG. 2C shows an example image sensor that includes different pixelgroups with different fields of view, in accordance with an embodimentof the disclosure.

FIG. 3 is a perspective view of example wearable glasses that include animage sensor and electronic components to facilitate human-computerinteraction, in accordance with an embodiment of the disclosure.

FIG. 4 illustrates an example block diagram that includes elements of adigital imaging device, in accordance with an embodiment of thedisclosure.

FIG. 5 illustrates an example block diagram and an example timingdiagram for generating a high dynamic range (“HDR”) image with an imagesensor, in accordance with an embodiment of the disclosure.

FIG. 6 illustrates an example block diagram and an example timingdiagram for generating high frame-rate video with an image sensor, inaccordance with an embodiment of the disclosure.

FIG. 7 illustrates an example block diagram and an example timingdiagram for generating HDR high-frame rate video with an image sensor,in accordance with an embodiment of the disclosure.

FIG. 8 illustrates an example block diagram and an example timingdiagram for generating an image with an expanded field of view with animage sensor, in accordance with an embodiment of the disclosure.

FIG. 9 illustrates an example block diagram and an example timingdiagram for generating a super-resolution image with an image sensor, inaccordance with an embodiment of the disclosure.

FIG. 10 illustrates an example lens system of a digital imaging devicethat includes different lenses focusing image light on different groupsof pixels of an image sensor, in accordance with an embodiment of thedisclosure.

FIG. 11 illustrates an example block diagram that includes differentlenses focusing image light on different groups of pixels of an imagesensor for generating after capture focused video, in accordance with anembodiment of the disclosure.

FIGS. 12A and 12B illustrate an example timing diagram and an exampleblock diagram that includes different lenses focusing image light ondifferent groups of pixels of an image sensor for generating an HDRafter capture focused image, in accordance with an embodiment of thedisclosure.

FIG. 13 illustrates an example filter system that filters image lightdirected toward different pixel groups of an image sensor, in accordancewith an embodiment of the disclosure.

FIG. 14 shows an example block diagram that illustrates different pixelgroups of an image sensor receiving image light through lenses andfilters and generating a light enhanced after capture focused image, inaccordance with an embodiment of the disclosure.

DETAILED DESCRIPTION

Embodiments of a system and method for capturing images are describedherein. In the following description, numerous specific details are setforth to provide a thorough understanding of the embodiments. Oneskilled in the relevant art will recognize, however, that the techniquesdescribed herein can be practiced without one or more of the specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures, materials, or operations are not shownor described in detail to avoid obscuring certain aspects.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments.

FIGS. 2A and 2B show example image sensors that include different pixelgroups (or arrays), in accordance with an embodiment of the disclosure.Image sensor 201 includes first pixel group 211, second pixel group 212,third pixel group 213, and fourth pixel group 214. In the illustratedembodiment, image sensor 201 is substantially rectangle shaped and isquartered into four sections and a pixel group occupies each of the foursections. In the illustrated embodiment, each pixel group is facingsubstantially the same direction. First pixel group 211, second pixelgroup 212, third pixel group 213, and fourth pixel group 214 may all bedisposed on the same semiconductor die.

Image sensor 202 includes first pixel group 220, second pixel group 230,third pixel group 240, and fourth pixel group 250. In the illustratedembodiment, each pixel group is facing substantially the same direction.In the illustrated embodiment, image sensor 202 is substantially shapedas an elongated rectangle divided into four sections with a pixel groupoccupying each section. First pixel group 220, second pixel group 230,third pixel group 240, and fourth pixel group 250 may all be disposed onthe same semiconductor die.

FIG. 2C shows an example image sensor that includes different pixelgroups with different fields of view, in accordance with an embodimentof the disclosure. Image sensor 203 includes first pixel group 221,second pixel group 222, third pixel group 223, and fourth pixel group224. In one embodiment, each pixel group has a 60 degree field of view,although different fields of view with different angles are possible. Inthe illustrated embodiment, first pixel group 221 has a field of viewfrom 0° to 60°, second pixel group 222 has a field of view of 20° to80°, third pixel group 223 has a field of view of 40° to 100°, andfourth pixel group 224 has a field of view of 60° to 120°. In thisembodiment, each pixel group has a field of view that overlaps anotherpixel group. Image sensor 203 may include four different semiconductordies that each include a pixel group and each semiconductor die may belaterally positioned at an angle relative to another semiconductor dieto give each pixel group a different field of view. It is appreciatedthat the field of view of each pixel group may be affected by lenses orfilters placed above the pixel groups.

In the illustrated embodiments of FIGS. 2A, 2B, and 2C, the pixel groupsin image sensors 201, 202, and 203 contain the same pixel count and arethe same dimension, but configurations with differing pixel counts anddiffering dimensions are possible. In one embodiment, the first pixelgroups, second pixel groups, third pixel groups, and fourth pixel groupshave pixel dimensions of common image resolutions (e.g. 640×480,1280×720, 1920×1080, etc.).

Many of the Figures presented in this disclosure illustrate imagesensors that include four pixel groups. In some embodiment, four or morepixel groups are required to capture the desired image, but in otherembodiments, image data from two or three pixel groups may be utilizedto generate images in a similar fashion as using image data from fourpixel groups. Thus, it is appreciated that some embodiments may bescaled to include more or less than the illustrated four pixel groups.

FIG. 3 is a perspective view of example wearable glasses 300 thatinclude an image sensor (e.g. image sensor 202 or 203) and electroniccomponents (e.g. controller 305) to facilitate human-computerinteraction, in accordance with an embodiment of the disclosure. In somecases, the wearable glasses are a head mounted display (“HMD”). An HMDis a display device worn on or about the head. HMDs usually incorporatesome sort of near-to-eye optical system to emit a light image within afew centimeters of the human eye. The illustrated embodiment of wearableglasses 300 includes lenses 345 disposed in frame 325 that includes lefttemple arm 330 and right temple arm 340. Although FIG. 3 illustrates atraditional eyeglass frame 325, embodiments of the present invention areapplicable to a wide variety of frame types and styles (e.g. visor,headband, goggles). Traditional eyeglass frame 325 may have a slim framethat does not include enough space to fix a traditional image sensorwith an acceptable resolution (e.g. 5 mega-pixels). An image sensor withan elongated shape such as image sensor 202 or image sensor 203 may bemore suitable for a slim eyeglass frame. Of course, an image sensor witha traditional shape or image sensor 201 may also be mounted on wearableglasses 300.

Although FIG. 3 illustrates an elongated image sensor (e.g. image sensor202 or 203), it appreciated that wearable glasses 300 may include anon-elongated image sensor, such as image sensor 201. In one embodiment,wearable glasses 300 includes image sensor 201 (a 2×2 array of pixelgroups). In one embodiment, image sensor 201 is mounted in the center ofwearable glasses 300, on or about the nose-piece. In one embodiment, twoimage sensors (e.g. image sensor 201, 202, or 203) are disposed onwearable glasses 300. In one example, one image sensor is disposed(forward facing) in the proximity of where left temple arm 330 meets theportion of frame 325 that secures lens 345 and second image sensor isdisposed (forward facing) in the proximity of where right temple arm 340meets the portion of frame 325 that secures lens 345.

Wearable glasses 300 may include a controller 305 and controller 305 mayinclude an integrated circuit with hardware, firmware, and/or softwarelogic. Controller 305 may be used to receive, transmit, and processdata. Controller 305 may receive and process image data and generateimages based on software algorithms. Controller 305 may be located in aplace or places other than in right temple arm 340.

FIG. 4 illustrates an example block diagram that includes elements of adigital imaging device, in accordance with an embodiment of thedisclosure. In the illustrated embodiment, first pixel group 420, secondpixel group 430, third pixel group 440, and fourth pixel group 450 arecoupled to first control circuitry 419, second control circuitry 429,third control circuitry 439, and fourth control circuitry 449,respectively. It is appreciated that, in one embodiment, first controlcircuitry 419, second control circuitry 429, third control circuitry439, and fourth control circuitry 449 may share electrical componentsand may be considered to be a single control circuitry module. In theillustrated embodiment, first control circuitry 419, second controlcircuitry 429, third control circuitry 439, and fourth control circuitry449 are coupled to receive first image capture signal 418, second imagecapture signal 428, third image capture signal 438, and fourth imagecapture signal 448, respectively.

Shutter controller 405 is coupled to transmit the image capture signals.Shutter controller 405 is coupled to initiate a first exposure period offirst pixel group 420 by sending first image capture signal 418 to firstcontrol circuitry 419, which facilitates the first exposure period.Shutter controller 405 also transmits image capture signals 428, 438,and 448 to control circuitry 429, 439, and 449 to facilitate second,third, and fourth exposure periods of the pixel groups 430, 440, and450, respectively.

In the illustrated embodiment, first readout circuitry 421 reads outfirst image data from first pixel group 420 and readout circuitry 431,441, and 451 function similarly to read out second, third, and fourthimage data from the second, third, and fourth pixel groups. Imageprocessing logic 490 receives first image 422, second image 432, thirdimage 442, and fourth image 452 from the respective readout circuitryfor further processing. Image processing logic 490 may include aprocessor and memory in order to edit, process, and combine image data.It is appreciated that, in one embodiment, first readout circuitry 421,second readout circuitry 431, and third readout circuitry 441 may shareelectrical components and may be considered to be a single readoutcircuitry module. In the illustrated embodiment, first pixel group 420,second pixel group 430, third pixel group 440, and fourth pixel group450 are disposed in image sensor 201 or 202 and the other circuitry isnot integrated into image sensor 201. In one embodiment, image sensor201 or 202 includes the illustrated control circuitry and readoutcircuitry. Other Figures in the disclosure may not specifically showcontrol circuitry and readout circuitry associated with each pixelgroup, but each pixel group may include control circuitry and readoutcircuitry, as described in the description of FIG. 4.

FIG. 5 illustrates an example block diagram and an example timingdiagram for generating a high dynamic range (“HDR”) image 595 with animage sensor, in accordance with an embodiment of the disclosure. FIG. 5shows first pixel group 520, second pixel group 530, third pixel group540, and fourth pixel group 550 facing substantially the same directionand receiving image light 503. First pixel group 520, second pixel group530, third pixel group 540, and fourth pixel group 550 may be examplesof the pixel groups shown in connection with image sensors 201 and 202.In the illustrated embodiment, first image capture signal 518 exposesfirst pixel group 520 to image light 503 for a first exposure periodthat has a duration that is less than the exposure periods of the otherpixel groups. Similarly, second image capture signal 528, third imagecapture signal 538, and fourth image capture 548 expose the respectivepixel groups for different durations. The image capture signals may comefrom shutter controller 405. In the illustrated embodiment, the first,second, third, and fourth exposure periods start at time T_(START),meaning images 522, 532, 542, and 552 are captured, at least in part,contemporaneously. If the photographer desires to capture a scene withmovement, the semi-contemporaneous image captures may be advantageouswhen compared with methods of generating HDR images that includecapturing images serially. Capturing images serially (one after theother) to produce an HDR image has the inherent risk of objects in thescene moving too quickly to effectively use HDR algorithms.

HDR algorithm logic 590 receive first image 522, second image 532, thirdimage 542, and fourth image 552 from the respective pixel groups andgenerates HDR image 595. After receiving the images, HDR algorithm logic590 may intelligently combine the images using known HDR methods togenerate HDR image 595. It is appreciated that HDR algorithm logic 590may need to perform certain other additional algorithms on first image522, second image 532, third image 542, and fourth image 552 to generateHDR image 595. For example, if the first, second, third, and fourthpixel groups face substantially the same direction and imagesubstantially the same scene or field of view, HDR algorithm logic 590may need to crop the received images to generate a HDR image 595 withthe same field of view.

FIG. 6 illustrates an example block diagram and an example timingdiagram for generating high frame-rate video with an image sensor, inaccordance with an embodiment of the disclosure. FIG. 6 shows firstpixel group 620, second pixel group 630, third pixel group 640, andfourth pixel group 650 facing substantially the same direction andreceiving image light 503. First pixel group 520, second pixel group530, third pixel group 540, and fourth pixel group 550 may be examplesof the pixel groups shown in connection with image sensors 201 and 202.In the illustrated embodiment, first image capture signal 618, secondimage capture signal 628, third image capture signal 638, and fourthimage capture signal 648 initiate exposure periods (in serial) for theirrespective pixel groups. In this embodiment, first pixel group 620captures an image, followed by second pixel group 630 capturing animage, followed by third pixel group 640 capturing an image, followed byfourth pixel group 650 capturing an image.

Image processing logic 690 receives these images from the pixel groupsand generates a high frame-rate video 695 using the images. Imageprocessing logic 690 is coupled to output high frame-rate video 695generated by interleaving at least a portion of pixel data received fromthe pixel groups. For example, first pixel data from first pixel group620, second pixel data from second pixel group 630, third pixel datafrom third pixel group 640, and fourth pixel data from fourth pixelgroup 650 may be interleaved to generate high frame-rate video 695. Itis appreciated that image processing logic 690 may need to performcertain other additional algorithms on the received images to generatehigh frame-rate video 695. For example, if the first, second, third, andfourth pixel groups face substantially the same direction and imagesubstantially the same scene or field of view, image processing logic690 may need to crop the received images to generate high frame-ratevideo 695. For example, image processing logic 690 may need to comparesecond pixel data with first pixel data and third pixel data todetermine a field of view commonality between the images beforefinalizing a 2^(nd) image in high frame-rate video 695.

Using two or more pixel groups to generate high frame-rate video ispotentially advantageous over using a single pixel group or pixel arrayto generate high frame-rate video because the frame rate of the videowill not necessarily be limited by pixel readout times of the pixelgroups. In one example, four pixel groups capture 30 frames per second.With image processing logic 690 interleaving the pixel data from eachpixel group, high frame-rate video could potentially be 120 frames persecond. If image sensor 201 or 202 had only three pixel groups capturingimages at 30 frames per second, image processing logic could interleavethe pixel data and potentially generate 90 frames per second highframe-rate video.

FIG. 7 illustrates an example block diagram and an example timingdiagram for generating HDR high-frame rate video with an image sensor,in accordance with an embodiment of the disclosure. FIG. 7 shows firstpixel group 620, second pixel group 630, third pixel group 640, andfourth pixel group 650 facing substantially the same direction andreceiving image light 503. In the illustrated embodiment, first imagecapture signal 718 exposes first pixel group 620 to image light 503 fora shorter duration than second image capture signal 728 exposes secondpixel group 630 to image light 503. Also in the illustrated embodiment,third image capture signal 738 exposes third pixel group 640 aftersecond pixel group 630 is exposed to image light 503. Third imagecapture signal 738 exposes third pixel group 640 to image light 503 fora shorter duration than fourth image capture signal 748 exposes fourthpixel group 650 to image light 503. Essentially, first pixel group 620and second pixel group 630 capture image data for a first compositeimage (e.g. first HDR image 756) and third pixel group 640 and fourthpixel group capture image data for a second composite image (e.g. secondHDR image 758) that is captured after the image data to generate thefirst composite image.

In the illustrated embodiment, first HDR processing logic 755 receivesfirst pixel data from first pixel group 620 and receives second pixeldata from second pixel group 630 and generates a composite image such asfirst HDR image 756. Second HDR processing logic 757 receives thirdpixel data from third pixel group 640 and receives fourth pixel datafrom fourth pixel group 650 and generates a composite image such assecond HDR image 758. Still referring to the illustrated embodiment,image processing logic 790 receives first HDR image 756 and second HDRimage 758 and interleaves the HDR images (or edited versions of the HDRimages) into HDR high frame-rate video 795. Image processing logic 790may need to perform additional algorithms on first HDR image 756 andsecond HDR image 758 to generate HDR high frame-rate video 795, such ascropping the received HDR images. It is appreciated that first HDRprocessing logic 755, second HDR processing logic 757 and imageprocessing logic 790 may be combined into one processor, a fieldprogrammable gate array (“FPGA”), or otherwise. It is also appreciatedthat additional algorithms may be performed on the first, second, third,and fourth pixel data prior to combining the pixel data into HDR imagesthat are included in HDR high frame-rate video 795.

FIG. 8 illustrates an example block diagram and an example timingdiagram for generating an image with an expanded field of view with animage sensor, in accordance with an embodiment of the disclosure. FIG. 8shows first pixel group 821, second pixel group 822, third pixel group823, and fourth pixel group 824 receiving image light 503. First pixelgroup 821 has a first field of view that partially overlaps a secondfield of view of second pixel group 822. Third pixel group 823 has athird field of view that partially overlaps the second field of view ofsecond pixel group 822 and fourth pixel group 824 has a fourth field ofview that partially overlaps the third field of view of third pixelgroup 823. In one example, each pixel group has an approximately 60°field of view. In one example, first pixel group 821 has a field of viewfrom 0° to 60°, second pixel group 822 has a field of view from 20° to80°, third pixel group 823 has a field of view from 40° to 100°, andfourth pixel group 824 has a field of view from 60° to 120°.

In the illustrated embodiment, image capture signals 818, 828, 838, and848 expose their respective pixel groups for exposure periods with asame duration. Shutter control circuitry (e.g. shutter controller 405)may be used to simultaneously initiate the exposure periods. Stitchingalgorithm logic 890 receives first image 826, second image 827, thirdimage 828, and fourth image 829 (from their respective pixel groups) andcombines the images to generate a composite image. In one embodiment,stitching algorithm logic 890 combines first image 826, second image827, third image 828, and fourth image 829 by stitching them togetherand outputs a panoramic image 895. Panoramic images may include 120° (ormore) field of view based on the combined field of view of the fourpixel groups. Since the exposure periods are initiated simultaneously,panoramic image 895 may include images capture contemporaneously, ratherthan images captured serially. It is appreciated that although theillustrated embodiment illustrates four pixel groups, the design couldalso be applied to two or three pixel groups to generate a panoramicimage.

FIG. 9 illustrates an example block diagram and an example timingdiagram for generating a super-resolution image with an image sensor, inaccordance with an embodiment of the disclosure. FIG. 9 shows firstpixel group 620, second pixel group 630, third pixel group 640, andfourth pixel group 650 facing substantially the same direction andreceiving image light 503. In the illustrated embodiment, image capturesignals 918, 928, 938, and 948 expose their respective pixel groups forexposure periods with a same duration. In the illustrated embodiment,super-resolution algorithm logic 990 receives first image 926 from firstpixel group 620 and receives second image 927 from second pixel group630. Super-resolution algorithm logic 990 receives third image 928 andfourth image 929 from third and fourth pixel group 640 and 650,respectively. Super-resolution algorithm logic 990 is coupled to outputa composite image (e.g. super resolution image 995) by performing asuper-resolution algorithm on the pixel data of the received images. Inthis way, super-resolution algorithm logic 990 generates a higherresolution image than the individual resolution of each of the pixelgroups.

In one embodiment, each of the pixel groups 620, 630, 640, and 650 areapproximately 1 mega-pixel (“MP”). However, an image with higherresolution than 1 MP can be generated by combining first image 926,second image 927, third image 928, and fourth image 929. Where a devicesuch as an HMD or wearable glasses 300 have limited real estate to placea higher resolution image sensor (e.g. 5 MP), it may still be able togenerate relatively high resolution images using an image sensor such asimage sensor 202.

FIG. 10 illustrates an example lens system of a digital imaging devicethat includes different lenses focusing image light on different groupsof pixels of an image sensor, in accordance with an embodiment of thedisclosure. Digital imaging device 1000 includes image sensor 1001,which includes first pixel group 1020, second pixel group 1030, thirdpixel group 1040, and fourth pixel group 1050. In the illustratedembodiment, first pixel group 1020, second pixel group 1030, third pixelgroup 1040, and fourth pixel group 1050 contain the same pixel count andare the same dimension, but configurations with differing pixel countsand differing dimensions are possible. In one embodiment, first pixelgroup 1020, second pixel group 1030, third pixel group 1040, and fourthpixel group 1050 have pixel dimensions of common image resolutions (e.g.640×480, 1280×720, 1920×1080, etc.).

Lens system 1002 includes first lens 1060, second lens 1070, third lens1080, and fourth lens 1090. First lens 1060 focuses image light 503 onfirst pixel group 1020. First lens light 1053 is the portion of imagelight 503 that travels through first lens 1060 and is focused on firstpixel group 1020. Second lens light 1063 is the portion of image light503 that travels through second lens 1070 and is focused on second pixelgroup 1030. Third lens light 1073 is the portion of image light 503 thattravels through third lens 1080 and is focused on third pixel group1040. And, fourth lens light 1083 is the portion of image light 503 thattravels through fourth lens 1090 and is focused on fourth pixel group1050. It is appreciated that each of first lens 1060, second lens 1070,third lens 1080, and fourth lens 1090 may include more than a singlelens, which may be aligned axially.

In the illustrated embodiment, first lens 1060 is configured to focus asubject approximately 20 centimeters away from image sensor 1001; secondlens 1070 is configured to focus subjects approximately two meters awayfrom image sensor 1001; third lens 1080 focuses subjects atapproximately 10 meters; and fourth lens 1090 focuses subjects atessentially infinity. Therefore, image sensor 1001 will be able to imagea scene with multiple depths of field that are centered around 20 cm,two meters, ten meters, and essentially infinity. First lens 1060 may beconfigured to facilitate reading QR codes and/or bar codes. It isappreciated that digital imaging device 1000 may have four differentdepths of field that converge or overlap. In one embodiment, first lens1060, second lens 1070, third lens 1080, and fourth lens 1090 aresubstantially the same, but have a different separation distance fromtheir respective pixel group, which provides different focus distancesto the different pixel groups.

Of course, different lenses that are configured to focus at distancesother than the distances specified above are possible. Similar imagingsystems may incorporate more or less lenses and pixel groups. In oneembodiment, lens system 1002 includes three lenses disposed over threepixel groups. In one embodiment, lens system 1002 includes only twolenses that focus image light 503 on two pixel groups.

In the illustrated embodiment, the pixels of the different pixel groupsborder or come very close to bordering pixels of other pixel groups.However, in some embodiments, pixels of the different pixel groups maybe separated by some distance, instead of bordering each other.

FIG. 11 illustrates an example block diagram that includes differentlenses focusing image light on different groups of pixels of an imagesensor for generating after capture focused video, in accordance with anembodiment of the disclosure. FIG. 11 shows first pixel group 1020,second pixel group 1030, third pixel group 1040, and fourth pixel group1050. First pixel group 1020, second pixel group 1030, third pixel group1040, and fourth pixel group 1050 may all measure lens light in responseto a common image capture signal, which may be sent from shuttercontroller 405. The measurements of lens light may be contemporaneous.

In response to the image capture signal, first pixel group 1020 measuresfirst lens light 1053 that has traveled through first lens 1060. Secondpixel group 1030 measures second lens light 1063, in response to theimage capture signal. Likewise third pixel group 1040 and fourth pixelgroup 1050 measure third lens light 1073 and fourth lens light 1083,respectively, in response to the image capture signal.

A first set of four images (e.g. first image 1122, second image 1132,third image 1142, and fourth image 1152) are captured by measuring thelens light and the images are sent to after capture focus (“ACF”) logic1190. ACF logic 1190 uses first image 1122, second image 1132, thirdimage 1142, and fourth image 1152 to generate a first ACF image. Shuttercontroller 405 may then initiate another image capture signal and thepixel groups may capture a second set of four images. ACF logic 1190 maythen use the second set of four images to generate a second ACF image.This process may be repeated so that a series of ACF images aregenerated (e.g. 1^(st) ACF image, 2^(nd) ACF image, 3^(rd) ACF image . .. ) and the series of ACF images can be made into ACF video 1195.

To generate each ACF image, ACF processing may be performed on firstimage 1122, second image 1132, third image 1142, and fourth image 1152.For example, ACF logic 1190 may analyze image data from the receivedimages to determine areas of the image data that are in focus. ACF logic1190 may save in memory or discard all or parts of the first and secondimage data based on the analysis. In one embodiment, the imageevaluation logic selects images based on the analysis of the focus andtransfers the image data to an additional processor (not shown) forfurther processing. Because two or more (e.g. four) images may becaptured simultaneously with different focus distances, the image datafrom the two or more images can be combined in a post-processingalgorithm (performed by ACF logic 1190), which may be performed in aprocessor. Given all of the image data (with different focus distances)collected of a scene, it is possible to construct an image that can berefocused with post-processing, after the images (e.g. 1122, 1132, 1142,and 1152) are captured. As with many of the embodiments presented inthis disclosure, the system shown in FIG. 11 may be modified from fourgroups of pixels and four lenses to include any number of groups ofpixels and corresponding lenses.

FIGS. 12A and 12B illustrate an example timing diagram and an exampleblock diagram that includes different lenses focusing image light ondifferent groups of pixels of an image sensor for generating an HDR/ACFimage, in accordance with an embodiment of the disclosure. FIG. 12Ashows first pixel group 1020, second pixel group 1030, third pixel group1040, and fourth pixel group 1050 receiving image light 503 throughfirst lens 1260, second lens 1270, third lens 1280, and fourth lens1290, respectively. First lens 1260 and second lens 1270 are configuredto focus image light 503 from a same distance (e.g. essentiallyinfinity) on first pixel group 1020 and second pixel group 1020. In oneembodiment, first lens 1260 and second lens 1270 are combined into onelens. Third lens 1280 and fourth lens 1290 are configured to focus imagelight 503 from a same distance (e.g. 10 meters) on third pixel group1040 and fourth pixel group 1050. In one embodiment, third lens 1280 andfourth lens 1290 are combined into one lens. First lens 1260 and secondlens 1270 may be configured to focus image light 503 at a certaindistance (e.g. essentially infinity) that is different from the distance(e.g. 10 meters) that third lens 1280 and fourth lens 1290 focus imagelight 503 by using different optical powers. First lens 1260 and secondlens 1270 may be configured to focus image light 503 at a certaindistance (e.g. essentially infinity) that is different from the distance(e.g. 10 meters) that third lens 1280 and fourth lens 1290 focus imagelight 503 by positioning the respective lenses at different distancesfrom the respective pixel groups.

FIG. 12B shows an example timing diagram that can be used in connectionwith FIG. 12A. In the illustrated embodiment, image capture signal 1218exposes first pixel group 1020 for a first duration that is shorter thana second duration that pixels of second pixel group 1030 are exposed toimage light 503 in response to image capture signal 1228. Also in theillustrated embodiment, image capture signal 1238 exposes third pixelgroup 1040 for a third duration that is shorter than a fourth durationthat pixels of fourth pixel group 1050 are exposed to image light 503 inresponse to image capture signal 1248. A shutter controller, such asshutter controller 405 may initiate the image capture signals.

In the illustrated embodiment, HDR/ACF image logic 1290 receives firstimage 1222 from first pixel group 1020 and receives second image 1232from second pixel group 1030. HDR/ACF image logic 1290 receives thirdimage 1242 and fourth image 1252 from third and fourth pixel group 1040and 1050, respectively. HDR/ACF image logic 1290 is coupled to output anHDR/ACF image 1295 by performing an HDR algorithm and an ACF algorithmon the pixel data received from the four images. Essentially, in theillustrated embodiment, first pixel group 1020 and second pixel group1030 are configured to generate a first HDR image at a first focusdistance (e.g. essentially infinity). Contemporaneously, third pixelgroup 1040 and fourth pixel group 1050 are configured to generate asecond HDR image at a second focus distance (e.g. 10 meters). In oneembodiment, after generating the first and second HDR image, HDR/ACFimage logic 1290 then performs an ACF algorithm on the two HDR images.The result is HDR/ACF image logic 1290 generating and outputting HDR/ACFimage 1295 that is an HDR image that has been focused after the imagecaptures.

FIG. 13 illustrates an example filter system that filters image lightdirected toward different pixel groups of an image sensor, in accordancewith an embodiment of the disclosure. FIG. 13 shows first pixel group1020 and third pixel group 1040 receiving image light 503 through acolor filter 1360. The portion of image light 503 that first pixel group1020 receives is first lens light 1353. The portion of image light 503that third pixel group 1040 receives is third lens light 1373. Secondpixel group 1030 and fourth pixel group 1050 receive image light 503through a black and white filter 1370. The portion of image light 503that second pixel group 1030 receives is second lens light 1363. Theportion of image light 503 that fourth pixel group 1050 receives isfourth lens light 1383. In one embodiment, black and white filter 1370is replaced with no filter, in other words, second pixel group 1030 andfourth pixel group 1050 do not receive image light 503 through a colorfilter. The absence of a color filter may give second pixel group 1030and fourth pixel group 1050 the ability to receive better light data(and possibly have higher dynamic range) in low light situations. Colorfilter 1360 may be a red/green/blue (“RGB”) filter or a Bayer filter.Color filter 1360 may be disposed much closer to the pixels of the pixelgroups than what is shown in FIG. 13.

Shutter control circuitry (e.g. shutter controller 405) may be coupledto the pixel groups and cause them to simultaneously measure light for asame exposure period. Image processing logic (e.g. image processinglogic 490) can receive each image generated by each of the pixel groupsand then generate a light-enhanced image. The image processing logic mayuse a light enhancement algorithm that takes light intensity data fromsecond pixel group 1030 and fourth pixel group 1050 and combines thelight intensity data with color data from first pixel group 1020 andthird pixel group 1040. Combining the light intensity data with thecolor data may give the light-enhanced image better dynamic range.

FIG. 14 shows an example block diagram that illustrates different pixelgroups of an image sensor receiving image light through lenses andfilters and generating a light-enhanced, after capture focused image, inaccordance with an embodiment of the disclosure. FIG. 14 shows firstpixel group 1020, second pixel group 1030, third pixel group 1040, andfourth pixel group 1050 receiving image light 503. First pixel group1020 receives image light 503 through first lens 1260 and color filter1360. Second pixel group 1030 receives image light 503 through secondlens 1270 and black and white filter 1370. Third pixel group 1040receives image light 503 through third lens 1280 and black and colorfilter 1360. Fourth pixel group 1050 receives image light 503 throughfourth lens 1290 and black and white filter 1370.

First lens 1260 and second lens 1270 are configured to focus image light503 from a same distance (e.g. essentially infinity) on first pixelgroup 1020 and second pixel group 1020. In one embodiment, first lens1260 and second lens 1270 are combined into one lens. Third lens 1280and fourth lens 1290 are configured to focus image light 503 from a samedistance (e.g. 10 meters) on third pixel group 1040 and fourth pixelgroup 1050. In one embodiment, third lens 1280 and fourth lens 1290 arecombined into one lens.

A shutter controller, such as shutter controller 405, may initiate imagecapture signals that expose each of the pixel groups (contemporaneously)for a same duration, generating first image 1422, second image 1432,third image 1442, and fourth image 1452. Light-enhanced/ACF logic 1490receives first image 1422, second image 1432, third image 1442, andfourth image 1452 from first pixel group 1020, second pixel group 1030,third pixel group 1040, and fourth pixel group 1050, respectively.

Light-enhanced/ACF logic 1490 is coupled to output a light-enhanced/ACFimage 1495. Light-enhanced/ACF logic may receive first image 1422 andsecond image 1432 and combine the pixel data to generate a firstlight-enhanced image. Light-enhanced/ACF logic may receive third image1442 and fourth image 1452 and combine the pixel data to generate asecond light-enhanced image. The first light-enhance image may befocused at a first distance (e.g. essentially infinity) from imagesensor 1001. The second light-enhanced image may be focused at a seconddistance (e.g. 10 meters) from image sensor 1001. Light-enhanced/ACFlogic 1490 may then perform an ACF algorithm (as described above) on thefirst light-enhanced image and the second light-enhanced image togenerate light-enhanced/ACF image 1495.

Referring back to FIG. 3, it may be advantageous to utilize at least oneflexible Printed Circuit Board (“PCB”) to build the embodimentsdescribed in FIGS. 4-14. In one embodiment, image processing logic 490(or logic 590, 690, 790, 890, 990, 1190, 1290, or 1490) and/or shuttercontroller 405 are disposed in left temple arm 330 or right temple arm340. A flexible PCB (a.k.a. PCB flex) may carry image capture signals,data, and clock signals to any of the image sensors (and theirrespective pixel groups) described in this disclosure. The shuttercontrol circuitry, image processing logic and image sensor(s) may beassembled on the same flexible PCB. Using a flexible PCB may beadvantageous because it may allow hinging of left temple arm 330 orright temple arm 340 while still maintaining connections to electronicsdisposed in the temple arms. In one embodiment, the flexible PCB ispartially (or fully) embedded in the frame of wearable glasses 300.Using flexible PCB may be advantageous because it may allow greaterdesign liberties (e.g. to have a curved frame) in creating wearableglasses 300. If more than one sensor is disposed in wearable glasses300, each of the sensors may be disposed on the same flexible PCB.

The processes or methods explained above are described in terms ofcomputer software and hardware. The techniques described may constitutemachine-executable instructions embodied within a tangible ornon-transitory machine (e.g., computer) readable storage medium, thatwhen executed by a machine will cause the machine to perform theoperations described. Additionally, the processes may be embodied withinhardware, such as an application specific integrated circuit (“ASIC”) orotherwise.

A tangible non-transitory machine-readable storage medium includes anymechanism that provides (i.e., stores) information in a form accessibleby a machine (e.g., a computer, network device, personal digitalassistant, manufacturing tool, any device with a set of one or moreprocessors, etc.). For example, a machine-readable storage mediumincludes recordable/non-recordable media (e.g., read only memory (ROM),random access memory (RAM), magnetic disk storage media, optical storagemedia, flash memory devices, etc.).

The above description of illustrated embodiments of the invention,including what is described in the Abstract, is not intended to beexhaustive or to limit the invention to the precise forms disclosed.While specific embodiments of, and examples for, the invention aredescribed herein for illustrative purposes, various modifications arepossible within the scope of the invention, as those skilled in therelevant art will recognize.

These modifications can be made to the invention in light of the abovedetailed description. The terms used in the following claims should notbe construed to limit the invention to the specific embodimentsdisclosed in the specification. Rather, the scope of the invention is tobe determined entirely by the following claims, which are to beconstrued in accordance with established doctrines of claiminterpretation.

What is claimed is:
 1. An imaging device comprising: a first pixel arrayarranged to capture a first image; a second pixel array arranged tocapture a second image, wherein the first pixel array and the secondpixel array are facing substantially a same direction; shutter controlcircuitry coupled to the first pixel array to initiate a first exposureperiod of the first pixel array to capture the first image and coupledto the second pixel array to initiate a second exposure period of thesecond pixel array to capture the second image; and processing logiccoupled to receive first pixel data of the first image and coupled toreceive second pixel data of the second image, wherein the processinglogic is configured to generate at least one image using the first pixeldata and the second pixel data.
 2. The imaging device of claim 1,wherein the first exposure period is for a first duration that isshorter than a second duration of the second exposure period.
 3. Theimaging device of claim 2, wherein the processing logic is coupled tooutput a composite image generated by combining the first pixel data andthe second pixel data.
 4. The imaging device of claim 3, wherein theprocessing logic is configured to execute a high dynamic range (“HDR”)algorithm to combine the first pixel data and the second pixel data intothe composite image.
 5. The imaging device of claim 1, wherein theshutter control circuitry is configured to initiate the second exposureperiod after the first exposure period starts.
 6. The imaging device ofclaim 5, wherein the processing logic is coupled to output a sequence ofimages generated by interleaving at least a first portion of the firstpixel data with at least a second portion of the second pixel.
 7. Theimaging device of claim 1, wherein the shutter control circuitry isconfigured to initiate the first exposure period and the second exposureperiod at substantially a same start time, and wherein the firstexposure period and the second exposure period is for substantially asame duration, the processing logic coupled to output a composite imagegenerated by performing a super-resolution algorithm on the first pixeldata and the second pixel data.
 8. The imaging device of claim 1 furthercomprising: a first color filter positioned to filter image lightpropagating toward the first pixel array, wherein the first color filteris not positioned to filter the image light from the second pixel array,and wherein the processing logic is coupled to output a light-enhancedimage generated by combining the first pixel data and the second pixeldata.
 9. The imaging device of claim 1 further comprising: a third pixelarray arranged to capture a third image; a fourth pixel array arrangedto capture a fourth image, wherein the third pixel array and the fourthpixel array are facing substantially the same direction as the firstpixel array, and wherein the shutter control circuitry is coupled toinitiate a third exposure period of the third pixel array and coupled toinitiate a fourth exposure period of the fourth pixel array, and whereinthe processing logic is coupled to receive third pixel data of the thirdimage and coupled to receive fourth pixel data of the fourth image,wherein the processing logic is configured to generate the at least oneimage using the first, second, third, and fourth pixel data.
 10. Theimaging device of claim 9, wherein the processing logic is configured tocombine the first pixel data and the second pixel data into a firstcomposite image and configured to combine the third pixel data and thefourth pixel data into a second composite image, and wherein theprocessing logic is configured to interleave the first composite imageand the second composite image into a video.
 11. The imaging device ofclaim 9 further comprising: a lens system configured to focus imagelight onto the first pixel array and the second pixel array from a firstfocus distance and configured to focus the image light onto the thirdpixel array and the fourth pixel array from a second focus distancedifferent from the first focus distance, wherein the first exposureperiod is shorter than the second exposure period and the third exposureperiod is shorter than the fourth exposure period, and wherein theprocessing logic is configured to perform a high dynamic range (“HDR”)algorithm on the first and second pixel data, configured to perform theHDR algorithm on the third and fourth pixel data, configured to performan after capture focusing (“ACF”) algorithm on the first and third pixeldata, and configured to perform the ACF algorithm on the second andfourth pixel data, the processing logic further configured to output anHDR/ACF image.
 12. The imaging device of claim 9 further comprising: acolor filter positioned to filter image light from the first and thethird pixel arrays, wherein the color filter array is not positioned tofilter the image light from the second and the fourth pixel arrays. 13.The imaging device of claim 9, wherein the shutter control circuitry isconfigured to stagger initiating the first, the second, the third, andthe fourth exposure periods, and wherein the processing logic isconfigured to interleave the first, the second, the third, and thefourth pixel data into a high-frame-rate video.
 14. The imaging deviceof claim 9 further comprising a head mounted display (“HMD”), whereinthe first pixel array, the second pixel array, the shutter controlcircuitry and the processing logic are disposed on the HMD, and aflexible Printed Circuit Board (“PCB”) embedded in a frame of the HMDcarries the first pixel data and the second pixel data to the processinglogic.
 15. An imaging device comprising: a first pixel array arranged tocapture a first image; a second pixel array arranged to capture a secondimage, wherein a first field of view of the first pixel array partiallyoverlaps a second field of view of the second pixel array; shuttercontrol circuitry coupled to simultaneously initiate a first exposureperiod of the first pixel array to capture the first image and a secondexposure period of the second pixel array to capture the second image;and processing logic coupled to receive first pixel data of the firstimage and coupled to receive second pixel data of the second image,wherein the processing logic is coupled to output a composite imagegenerated by stitching together the first pixel data and the secondpixel data.
 16. The imaging device of claim 15, wherein the firstexposure period and the second exposure period are the same duration.17. The imaging device of claim 15 further comprising: a third pixelarray arranged to capture a third image, wherein a third field of viewof the third pixel array partially overlaps the second field of view ofthe second pixel array; wherein the shutter control circuitry is coupledto initiate a third exposure period of the third pixel array to capturethe third image, the first, second, and third exposure periods initiatedsimultaneously, and wherein the processing logic is coupled to receivethird pixel data of the third image and coupled to output the compositeimage generated by stitching together the first, second, and third pixeldata, the composite image substantially including a combined field ofview of the first, second, and third fields of view.
 18. The imagingdevice of claim 15 further comprising a head mounted display (“HMD”),wherein the first pixel array, the second pixel array, the shuttercontrol circuitry and the processing logic are disposed on the HMD, anda flexible Printed Circuit Board (“PCB”) embedded in a frame of the HMDcarries the first pixel data and the second pixel data to the processinglogic.
 19. A method comprising: initiating a first exposure period of afirst pixel array to capture a first image; initiating a second exposureperiod of a second pixel array to capture a second image, wherein thefirst pixel array and the second pixel array are disposed on a samesemiconductor die and are facing substantially a same direction; andgenerating at least one image based on first pixel data from the firstimage and second pixel data from the second image.
 20. The method ofclaim 19 further comprising: initiating a third exposure period of athird pixel array to capture a third image; initiating a fourth exposureperiod of a fourth pixel array to capture a fourth image, wherein thethird pixel array and the fourth pixel array are disposed on the samesemiconductor die and are facing substantially the same direction as thefirst pixel array and the second pixel array, and wherein generating theat least one image is also based on third pixel data from the thirdimage and fourth pixel data from the fourth image.
 21. The method ofclaim 20, wherein the first exposure period is for a first duration thatis shorter than a second duration of the second exposure period and thethird exposure period is for a third duration that is shorter than afourth duration of the fourth exposure period, and wherein generatingthe at least on image includes generating a first composite image withthe first and second pixel data and generating a second composite imagewith the third and fourth pixel data and interleaving the firstcomposite image with the second composite image into a sequence ofimages.
 22. The method of claim 20, wherein the first, second, third,and fourth exposure periods are staggered, and wherein generating the atleast one image includes interleaving the first, second, third, andfourth pixel data into a high-frame-rate video.
 23. The method of claim20, wherein the first pixel array and the second pixel array receiveimage light focused at a first optical power and the third pixel arrayand the fourth pixel array receive the image light focused at a secondoptical power, and wherein the first exposure period is for a firstduration that is shorter than a second duration of the second exposureperiod and the third exposure period is for a third duration that isshorter than a fourth duration of the fourth exposure period, thegenerating the at least one image including performing a high dynamicrange (“HDR”) algorithm and an after capture focusing (“ACF”) algorithm.24. The method of claim 19, wherein the first exposure period is for afirst duration that is shorter than a second duration of the secondexposure period and wherein generating the at least one image includesgenerating a high dynamic range (“HDR”) image.
 25. The method of claim19, wherein the first exposure period and the second exposure period arestaggered, and wherein generating the at least one image includesinterleaving at least a first portion of the first pixel data and asecond portion of the second pixel data to generate high-frame-ratevideo.
 26. The method of claim 19, wherein generating the at least oneimage includes generating a composite image by performing asuper-resolution algorithm on the first pixel data and the second pixeldata.
 27. The method of claim 19, wherein generating the at least oneimage includes generating a light-enhanced image by combining color datafrom the first pixel data and intensity data from the second pixel data,wherein the first pixel array receives image light through a colorfilter and the second pixel array does not receive the image lightthrough the color filter.